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Introduction 

U.S. teacher educators are faced with serious challenges to 
demonstrate the quality of the graduates they prepare. These 
challenges are expressed in the country' s fixation on 
accountability and testing. We are witnessing the growth of 
standardized tests and alternative routes to certification as 
panaceas intended to solve the teacher shortage crisis in most 
states . National accreditation and state program approval agencies 
are attempting to stem the tide through the ever-increasing demand 
for standards-based assessment data documenting teacher quality. 

The common set of standards used nationally for accreditation was 
developed by the Council of Chief State School Officers and 
promulgated by the Interstate New Teacher Assessment and Support 
Consortium (INTASC) in the form of ten principles. Each of the 
principles includes indicators written at the knowledge, skill, and 
dispositional levels, forming constructs that colleges are required 
to measure. 

Sadly, neither the profession nor the accreditors have 
realized the need for objective measurement. This is largely a 
function of what Stiggins bemoans as assessment illiteracy (2000) . 
They are satisfied, at best, with ordinal scales for poorly 
constructed criteria on ill-defined tasks, or, at worst, with 
counting papers in portfolios constructed without regard to any 
form of psychometric consideration (Wilkerson and Lang, 2003) . 

In this paper, however, the focus was on assessment of 
dispositions. Searches for measurement of this elusive construct 
have been fruitless, and teacher educators are clamouring for 
answers to this requirement since it is being imposed by the 
National Council for the Accreditation of Teacher Educators 
(NCATE) . At a workshop these authors conducted last year for the 
American Association of Colleges of Teacher Education (Wilkerson, 
et al . , 2003), virtually all participants saw dispositions as their 
burning issue. 

The measurement community has been largely uninvolved in 
trying to solve the disposition dilemma. Teacher educators who 
have attempted disposition assessment generally resort to one or 
more disconnected measures with single item structures. These are 
often poorly conceptualized, failing to adequately sample from the 
construct to be measured. This paper presents an attempt at 
providing an objective measurement solution to the problem. The 
approach to this problem started by envisioning multiple 
instruments, of different formats and at differing levels of 
inference. The construct map included the content from the INTASC 
principles and was framed by the stages of affective response 
described by Krathohl's Taxonomy. As exploration of the results by 
sub-grouping items as confounding or informing the results was the 
primary question of this paper. The original plan includes six 
instrument types, but three are considered here. Comments on 
planned Rasch models and dimensionality: 
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As described by Wright and Stone (1999, "The way to proceed in 
understanding multidimensionality is first to construct 
unidimensional variables upon which to make measures and then 
evaluate the relationship among the measures generated from these 
singular variables." (p . 181) . 

"Confusing a number of attributes into a single generic score makes 
confident predictions from that score more hazardous and the score 
a less useful summary of ability or achievement. But carefully 
constructed tests that make good measurement estimates of single 
attributes might be sufficient for a number of thoughtfully decided 
purposes. For special or difficulty situations, collecting 
additional estimates of other appropriate attributes is essential." 
(Bond & Fox, 2007, p. 34) . 

"The art of assessing dimensionality is to find the smallest number 
of ability domains such that the are both statistically well- 
defined and substantively meaningful." (Briggs & Wilson, 2004, p. 
323) . 



Standards based assessment of the INTSAC principles must 
consider the ten principles as subscores at the least. 

Dispositions are affective constructs, so high inference items 
types require extensive scoring, while self-report is subject to 
faking. For our purposes, the way to achieve confidence in the 
assessment was to use multiple item types that confirm the score 
validity. This mandated that the INTASC principle construct needed 
a framework so that scoring for both dichotomous and various rating 
conditions was meaningful as an attribute and statistically 
reasonable. In this pilot, we explored the characteristics of the 
initial calibration of the new instruments to assess teacher 
dispositions . 



Method 



Instruments 

This was a field test of three instruments that make use of 
three different item structures (agree/disagree statements, self- 
report questions, and apperception) . The total scale to be created 
combines all instruments. The belief scale is composed of 60 
items, with sub-scales for each of the ten INTASC Principles. The 
self report questionnaire contains ten questions, also aligned to 
INTASC Principles. The apperception instrument presents 20 
stimulus cards, two each for the 10 INTASC principles. The 
instruments range from low to high inference. The item design and 
scoring guides were developed as framed by the Krathwohl Affective 
Taxonomy. The three instruments considered here are the Belief 
Scale Aligned with Teacher Standards (BATS) , Experiential Teacher 
Questionnaire (ETQ) , and the Situational Reflection Assessment 
( SRA) . 
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Sample 

At present, we have analysed data on 335 respondents from the 
teacher education programs at three colleges. With a few missing 
cases, teacher candidates ranging from pre-internship to final 
internship to inservice graduate students were assessed. The 
sample came from teacher education programs at three universities 
in Florida. 



Analysis 



We are using three Rasch models for the scales: dichotomous, 
rating scale, and FACETS as follows (Stone, 2003; Linacre, 2007) : 
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Results 

The following (Figure 1) is the logistic ruler produced 
from the initial analysis. The 60 Thurstone (agree-disagree) 
statements were scored dichotomously and are indicated as a D. 
The two rated instrument items are indicated by an X. In this 
calibration, all items were pooled as one scale and coded as 
item groups in Winsteps . 

That is followed by the Separation Table from Winsteps 
(Table 2) . The OUTFIT ZSTD of -0.3 and standard deviation of 
1.9 imply more variability than expected, but Linacre (2007) 
warns that "the fit statistics stratify by item type; so that, 
say, all the Yes-No items overfit, and all the Frequency items 
underfit" (p. 89) . Figure 1 is a from a joint calibration of 
the data. 

Table 3 contains the INTASC Principle statistics across 
all three instruments in the battery. In this case, each 
principle is represented by 6 Thurstone items, 1 questionnaire 
item, and 2 apperception items. The questionnaire and 
apperception items are scored on a rating of 0 to 6 based on 
modification of the Krathwohl affective taxonomy. The ratings 
are : 

0=Unaware 
l=Receiving 
2=Responding 
3=Valuing 
4=Organiz ing 
5=Characteriz ing 
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TABLE 1.1 Disposition Analysis Spring 08 Pilot G ZOU920WS.TXT Mar 20 10:30 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 
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Figure 1. Map of Items and Persons for BATS, ETQ, and SRA 
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Table 1. Summary Statistics for BATS, ETQ, and SRA as 
Separate Calibrations 

TABLE 3.1 Disposition Analysis Spring 08 Pilot G ZOU920WS.TXT Mar 20 10:30 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 



SUMMARY OF 261 MEASURED (NON-EXTREME) persons 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



MEAN 68.0 72.3 58.08 2.57 1.16 .5 .96 -.3 
S.D. 18.6 5.8 9.28 .69 .70 2.5 .64 1.9 
MAX. 110.0 81.0 89.58 10.33 5.65 9.9 4.27 9.6 
MIN. 12.0 60.0 24.07 1.94 .39 -3.4 .27 -3.0 



| REAL RMSE 3.00 ADJ.SD 8.78 SEPARATION 2.93 person RELIABILITY .90 | 

| MODEL RMSE 2.66 ADJ.SD 8.89 SEPARATION 3.34 person RELIABILITY .92 | 

| S.E. OF person MEAN = .58 I 

+ + 

MINIMUM EXTREME SCORE: 74 persons 

VALID RESPONSES: 80.3% 

SUMMARY OF 335 MEASURED (EXTREME AND NON-EXTREME) persons 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



| MEAN 52.9 69.6 41.94 6.05 I 

I S.D. 32.6 7.2 31.40 6.56 I 

I MAX. 110.0 81.0 89.58 18.31 I 

| MIN. .0 60.0 -14.98 1.94 I 

I I 

| REAL RMSE 9.00 ADJ.SD 30.08 SEPARATION 3.34 person RELIABILITY .92 | 

| MODEL RMSE 8.92 ADJ.SD 30.10 SEPARATION 3.37 person RELIABILITY .92 | 

| S.E. OF person MEAN =1.72 I 

+ + 

person RAW SCORE-TO-MEASURE CORRELATION = .94 (approximate due to missing data) 

CRONBACH ALPHA (KR-20) person RAW SCORE RELIABILITY = .97 (approximate due to missing data) 

SUMMARY OF 90 MEASURED (NON-EXTREME) items 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



MEAN 197.1 209.6 50.00 1.64 .99 -.1 1.18 .8 
S.D. 51.1 73.9 14.40 .48 .36 2.5 .83 3.8 
MAX. 309.0 261.0 74.52 2.55 2.12 5.9 4.00 9.9 
MIN. 50.0 52.0 28.97 .90 .50 -3.2 .19 -4.3 



| REAL RMSE 1.77 ADJ.SD 14.30 SEPARATION 8.09 item RELIABILITY .98 | 

| MODEL RMSE 1.71 ADJ.SD 14.30 SEPARATION 8.39 item RELIABILITY .99 | 

| S.E. OF item MEAN =1.53 I 

+ + 

UMEAN=50 .000 USCALE=10 . 000 

item RAW SCORE-TO-MEASURE CORRELATION = -.53 (approximate due to missing data) 
18862 DATA POINTS. APPROXIMATE LOG-LIKELIHOOD CHI-SQUARE: 20432.72 
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Table 2 . Results of the Calibration by Principle and 
Instrument Type 



TABLE 27.1 Disposition Analysis Spring 08 Pilot ZOU034WS.TXT Mar 20 10:38 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 



Subtotal specification is: ISUBTOTAL=$S8W2 
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The INTASC Principles are coded as : 

1 . Content Knowledge 

2 . Development and Learning 

3 . Diversity 

4 . Critical Thinking 

5. Learning Environment and Motivation 

6. Communication 

7 . P lanning 

8 . Assessment 

9. Reflection and Continuous Improvement 

10. Collegiality and Professionalism 

TABLE 27.14 Disposition Analysis Spring 08 Mar 20 10:38 2008 

INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 

Subtotal specification is: ISUBT0TAL=$S1W3 
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Table 3 . Summary of the Calibration for Each Instrument Type 

TABLE 27.17 Disposition Analysis Spring 08 Pilot ZOU034WS.TXT Mar 20 10:38 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 



"BAT" SUBTOTAL FOR 60 NON-EXTREME items 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



MEAN 201.6 261.0 42.00 1.88 .80 -1.3 .74 -1.4 
S.D. 43.7 .0 10.71 .39 .19 1.7 .27 1.5 
MAX. 241.0 261.0 74.52 2.55 1.19 5.0 1.19 2.1 
MIN. 50.0 261.0 28.97 1.32 .50 -3.2 .19 -4.3 



| REAL RMSE 1.92 ADJ.SD 10.53 SEPARATION 5.48 item RELIABILITY .97 | 

| MODEL RMSE 1.92 ADJ.SD 10.54 SEPARATION 5.50 item RELIABILITY .97 | 

| S.E. OF item MEAN =1.39 I 

| MEDIAN =38.14 I 

+ + 

TABLE 27.18 Disposition Analysis Spring 08 Pilot ZOU034WS.TXT Mar 20 10:38 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 



"ETQ" SUBTOTAL FOR 10 NON-EXTREME items 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



MEAN 264.3 134.6 66.27 .91 1.22 1.8 1.91 5.6 
S.D. 27.1 1.4 2.10 .00 .20 1.4 .47 2.1 
MAX. 309.0 137.0 69.15 .92 1.72 5.2 3.00 9.9 
MIN. 228.0 133.0 62.98 .90 1.02 .2 1.36 2.8 



| REAL RMSE 1.01 ADJ.SD 1.84 SEPARATION 1.83 item RELIABILITY .77 | 

| MODEL RMSE .91 ADJ.SD 1.89 SEPARATION 2.08 item RELIABILITY .81 | 

| S.E. OF item MEAN = .70 I 

| MEDIAN =66.27 I 

+ + 

TABLE 27.19 Disposition Analysis Spring 08 Pilot ZOU034WS.TXT Mar 20 10:38 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 90 items 13 CATS 3.64.2 



" SRA" SUBTOTAL FOR 20 NON-EXTREME items 

+ + 

| RAW MODEL INF IT OUTFIT | 

| SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD | 



MEAN 149.9 92.8 65.87 1.28 1.45 2.8 2.16 5.2 
S.D. 32.9 13.3 3.31 .13 .34 1.8 .96 3.2 
MAX. 202.0 105.0 72.83 1.67 2.12 5.9 4.00 9.9 
MIN. 69.0 52.0 57.97 1.15 1.04 .4 1.04 .4 



| REAL RMSE 1.57 ADJ.SD 2.92 SEPARATION 1.86 item RELIABILITY .77 | 

| MODEL RMSE 1.29 ADJ.SD 3.05 SEPARATION 2.37 item RELIABILITY .85 | 

| S.E. OF item MEAN = .76 I 

| MEDIAN =65.48 I 

+ + 
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Table 4. Item Statistics for BATS, ETQ, and SRA 

TABLE 10.1 Disposition Analysis Spring 08 Pilot ZOU604WS.TXT Mar 20 10:58 2008 
INPUT: 335 persons 90 items MEASURED: 335 persons 60 items 2 CATS 3.64.2 

person: REAL SEP.: 1.35 REL . : .65 ... item: REAL SEP.: 4.72 REL . : .96 

"BATS" item STATISTICS: MISFIT ORDER 



+ + 
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203 


243 


55.30 


1.801 


. 94 


-.5 1 


. 84 


-1.0 


1 q 


,79| 


84 . 4 


83.91 


BAT58RP07 


1 


| 12 


235 


243 


36.82 


3.65 1 


. 94 


-.11 


. 61 


-.9 


IP 


. 94 | 


96.7 


96.7| 


BAT12RC07 


1 


| 11 


241 


243 


22 .46 


7.131 


.93 


. 1 1 


.39 


-.7 


1 O 


. 98 | 


99.2 


99.2| 


BAT11VA01 


1 


1 8 


236 


243 


35.40 


3.89 1 


. 93 


-.11 


.57 


-.9 


1 n 


. 95 | 


97 . 1 


97.1| 


BAT080R03 


1 


I 46 


225 


243 


45.70 


2.51| 


. 92 


-.3 


. 68 


-1.2 


1 m 


. 89 | 


92.6 


92.61 


BAT46VA01 


1 


1 51 


232 


243 


40.24 


3.14| 


. 92 


-.2| 


.71 


-.7 


1 1 


. 93 | 


95.5 


95.51 


BAT51RC0 6 


1 


35 


241 


243 


22 .46 


7.131 


. 92 


. 1 1 


.20 


-1.3 


1 k 


. 98 | 


99.2 


99.2| 


BAT35RC03 


1 


1 25 


235 


243 


36.82 


3.651 


. 92 


-.11 


.50 


-1.3 


1 j 


. 95 | 


96.7 


96.7| 


BAT25VA0 6 


1 


1 50 


224 


243 


46.31 


2.45| 


. 91 


-.4| 


.78 


-.8 


| i 


. 89 | 


92.6 


92.2| 


BAT50RC05 


1 


1 6 


241 


243 


22 .46 


7.131 


. 90 


. 1 1 


. 14 


-1.5 


1 h 


. 98 | 


99.2 


99.2| 


BAT06RP04 


1 


| 40 


238 


243 


31.88 


4.561 


.90 


-.11 


.42 


-1.2 


1 g 


. 96 | 


97.9 


97.91 


BAT40RC02 


1 


32 


238 


243 


31.88 


4.561 


. 90 


-.11 


.45 


-1.1 


l f 


. 96 | 


97.9 


97.91 


BAT 3 2 RP 02 


1 


| 54 


229 


243 


42.88 


2.81| 


. 89 


-.4| 


. 65 


-1.1 


1 e 


. 92 | 


94.2 


94.2| 


BAT54RC0 9 


1 


1 59 


231 


243 


41.19 


3.02| 


. 88 


-.4| 


.56 


-1.4 


1 d 


. 93 | 


95.1 


95.11 


BAT59RC09 


1 


1 15 


238 


243 


31.88 


4.561 


. 87 


-•2| 


.30 


-1.6 


1 c 


. 97 | 


97.9 


97.91 


BAT15RC08 


1 


I 55 


232 


243 


40.24 


3.14| 


. 87 


-.4| 


.49 


-1.6 


lb 


. 93 | 


95.5 


95.5 1 


BAT55RC01 


1 


I 18 


240 


243 


26.60 


5.85| 


. 87 


-.11 


. 15 


-1.8 


1 a 


. 98 | 


98.8 


98.8 1 


BAT18RC09 


1 






























































| MEAN 


201.6 


243.0 


50.00 


2.73| 


. 99 


.2 | 


. 87 


.0 




1 


85.9 


86.2| 






I S.D. 


43.7 


.0 


15.17 


1.531 


.07 


. 9 


.29 


1.2 




1 


12.2 


11.61 
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Table 4 . (Continued) 

TABLE 10.1 Disposition Analysis Spring 08 Pilot ZOU146WS.TXT Mar 20 11:00 2008 
INPUT: 335 persons 90 items MEASURED: 137 persons 10 items 6 CATS 3.64.2 



person: REAL SEP.: 2.13 REL.: .82 ... item: REAL SEP.: 2.22 REL . : .83 

"ETQ" item STATISTICS: MISFIT ORDER 



+ + 

| ENTRY RAW MODEL | INFIT | OUTFIT |PTMEA| EXACT MATCH | | 

INUMBER SCORE COUNT MEASURE S.E. | MNSQ ZSTD | MNSQ ZSTD|CORR.| OBS% EXP% | item G | 

| + + + + + | 



| 67 273 135 49.26 .99|1.40 3.1|1.37 2.9|A . 67 | 37.0 42. 9| ETQ07**07 2 | 
| 66 258 133 50.19 .99|1.07 .7|1.11 1.0|B . 45 | 43.6 42. 9| ETQ06**06 2 | 
| 65 301 136 46.78 .99|1.06 .5|1.06 .6|C . 64 | 45.6 43. 1| ETQ05**05 2 | 
| 70 265 133 49.88 .99|1.01 .1|1.01 .2|D . 65 | 49.6 43. 2| ETQ10**10 2 | 
| 61 238 136 52.62 .99|1.01 .1| .99 .0|E . 69 | 44.1 42. 6| ETQ01**01 2 | 
| 64 251 135 51.38 . 99 | .92 -.7| .91 -.7|e . 66 | 48.1 42. 5| ETQ04**04 2 | 
| 68 228 134 53.35 . 99 | .92 -.7| .90 -.8|d . 62 | 36.6 42. 5| ETQ08**08 2 | 
| 69 231 133 52.94 1.00| .87 -1 . 2 | .87 -1 . 1 | c . 68 1 48.9 42. 4| ETQ09**09 2 | 
| 63 289 134 47.54 . 99 | .86 -1 . 2 | .86 -1.2|b . 68 | 48.5 43. 2| ETQ03**03 2 | 
| 62 309 137 46.07 . 98 | .85 — 1 . 3 | .85 -1 . 3 | a . 69 | 51.8 43. 3| ETQ02**02 2 | 
| + + + + + | 



| MEAN 264.3 134.6 50.00 .99|1.00 -.1| .99 -.1| | 45.4 42. 9| | 

| S.D. 27.1 1.4 2.47 . 00 | .15 1.2| .15 1 . 2 | | 4.9 .3| | 

+ + 



TABLE 10.1 Disposition Analysis Spring 08 Pilot ZOU854WS.TXT Mar 20 10:56 2008 
INPUT: 335 persons 90 items MEASURED: 193 persons 20 items 5 CATS 3.64.2 



person: REAL SEP.: 1.55 REL.: .71 ... item: REAL SEP.: 2.04 REL.: .81 

"SRA" item STATISTICS: MISFIT ORDER 



+ 

| ENTRY 
| NUMBER 


RAW 

SCORE 


COUNT 


MEASURE 


MODEL | INFIT | OUTFIT | PTMEA | 

S.E. | MNSQ ZSTD | MNSQ ZSTD|CORR.| 


EXACT 

OBS% 


MATCH | 
EXP% | 


item 


+ 

G | 


























| 88 


69 


52 


52.60 


1.7311.51 


2.611.45 


2 . 1 | A 


.41 | 


34 . 6 


44.91 


SRA18**09 


3 I 


1 76 


192 


105 


46.89 


1.2311.47 


3.2|1.40 


2.8 |B 


.491 


41 . 9 


45.3 1 


SRA06**03 


3 I 


1 89 


89 


87 


59.00 


1.3811.35 


2.311.25 


1.7 |C 


.45 | 


43.7 


44.1| 


SRA19**10 


3 


82 


166 


105 


50.75 


1.21|1.23 


1.811.22 


1 . 6 | D 


.58 | 


39.0 


44.61 


SRA12**06 


3 


90 


159 


105 


51.79 


1.21|1.14 


1.111.14 


1.1 |E 


.47 | 


42 . 9 


44.3 1 


SRA20**10 


3 I 


| 78 


169 


105 


50.31 


1.21|1.11 


.911.13 


1 . 0 | F 


.48 | 


54.3 


44.61 


SRA08**04 


3 


| 84 


202 


104 


45.02 


1.2511.13 


1.011.10 


.8 |G 


. 64 | 


40 . 4 


46.4| 


SRA14**07 


3 


I 86 


165 


105 


50.89 


1.21|1.03 


.311.07 


. 6 | H 


.57 | 


41 . 9 


44.6| 


SRA16**08 


3 


| 71 


141 


100 


51.84 


1.24|1.06 


.511.05 


.4 11 


.431 


50.0 


44.3 1 


SRA01**01 


3 I 


80 


190 


104 


46.87 


1.2311.02 


.211.05 


.4 | J 


.54 | 


39.4 


45.3 1 


SRA10**05 


3 1 


| 74 


189 


105 


47.34 


1.22|1.02 


■2| .99 


.01 j 


. 62 | 


45.7 


45.4| 


SRA04**02 


3 I 


| 72 


166 


103 


50.36 


1.22| .93 


-.51 .98 


-. 1 | i 


.57 | 


46.6 


44.7| 


SRA02**01 


3 I 


| 87 


116 


84 


53.40 


1.34| .89 


-.81 .87 


— . 9 l h 


.58 | 


50.0 


43.51 


SRA17**09 


3 


| 77 


141 


85 


48.97 


1.331 .82 


-1.3| .81 


-1.4 |g 


. 65 | 


48.2 


44.4| 


SRA07**04 


3 I 


I 83 


163 


83 


44.70 


1.381 .80 


-1.4| .82 


-1.3 | f 


.531 


55.4 


45.0| 


SRA13**07 


3 


I 85 


140 


83 


48.86 


1.351 .81 


-1.4| .81 


-1.4 | e 


. 64 | 


47 . 0 


44.5| 


SRA15**08 


3 I 


I 81 


143 


84 


48.41 


1.34| .78 


-1.7 .79 


-1.5 | d 


.731 


52 . 4 


44.4| 


SRA11**06 


3 


79 


128 


86 


51.65 


1.32| .70 


-2.5| .69 


-2.51c 


. 63 | 


53.5 


43.7 1 


SRA09**05 


3 I 


1 75 


130 


86 


51.30 


1.32| .61 


-3.3| .62 


-3.1 |b 


. 60 | 


55.8 


43.7 1 


SRA05**03 


3 1 


73 


139 


85 


49.04 


1.331 .58 


-3.5| .59 


-3.4 | a 


.50 | 


57.6 


44.2| 


SRA03**02 


3 1 


















































| MEAN 


149.9 


92.8 


50.00 


1.3011.00 


-.11 .99 


-.2 | 


i 


47 . 0 


44.6| 






| S.D. 

+ 


32.9 


13.3 


3.14 


.12| .25 


1.8| .23 


1.7| 


i 


6.3 


. 6 | 




+ 




Category Information 
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61. ETQ01**01 




Measure relative to item difficulty 



0 1 2 3 4 5 

Figure 2 . Threshold for the ETQ Rating Scale based on Krathwohl 

0=Unaware 

l=Receiving 

2=Responding 

3=Valuing 

4=Organizing 

5=Character izing 
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71.SRA01**01 




Measure relative to item difficulty 



0 12 3 4 

Figure 3 . Threshold for the SRA Rating Scale based on Krathwohl 

0=Unaware 
l=Receiving 
2=Responding 
3=Valuing 
4=Organizing 
5=Characterizing 
(No 5'S observed) 
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Disposition Pilot FACETS Analysis 03-20-2008 12:34:13 
Table 6.0 All Facet Vertical "Rulers". 



Vertical = (1*,2A, 3A) Yardstick (columns , lines , low, high) = 0,10, -3, 2 



| Measr | +Teacher 
1 1 


| -Instrument | -INTASC 
Type | Principle 




i 


Krathwohl | 
| Rating 


+ 2 + 
1 1 

1 1 

1 1 

1 1 


+ + 

i i 

i i 

i i 

i i 

i i 






+ (5) + 

1 4 | 

1 1 

1 1 

1 1 

1 1 


1 1 

1 1 

+ 1 + . 


i i 

i i 

i i 

i i 

+ SRA + 

1 1 






1 1 

1 1 

1 1 

+ + 

1 1 


1 1 *• 

| | **. 

1 1 **** 


1 1 

1 etq | 

1 1 

1 1 






1 1 

1 1 

1 1 

1 3 | 


1 1 *** # 

1 1 **** 


| | Reflection 

1 1 






1 1 
1 1 


1 1 ***** # 

1 1 ****** 

* Q * ******** 


| | Critical Thinking 

| | Assessment 

★ * 


Professionalism 


Subject Matter 


1 1 

1 1 

* * 


1 1 ***** # 

1 1 ****** _ 

1 I *** # 


| | Diversity 

| | Communications 

i i 


En vi r onment 


Planning 


1 1 
1 1 


1 I ***** # 

1 1 **** _ 

1 1 *. 

1 1 *• 

1 1 ** 

1 1 * 


| | Learning 

i i 

i i 

i i 

i i 

i i 






1 1 

1 1 

1 1 

1 2 | 
1 1 

1 1 


+ -1 + . 
1 1 


+ + 

i i 

i i 

i i 






+ + 

1 1 

1 1 


1 1 
1 1 
1 1 
1 1 
1 1 


i i 

i i 

i i 

| BATS | 

1 1 






1 1 
1 1 
1 1 
1 1 
1 1 1 


1 1 

+ -2 + 
1 1 

1 1 

1 1 

1 1 


1 1 

+ + 

1 1 

1 1 

1 1 

1 1 






1 1 

+ + 

1 1 

1 1 

1 1 

1 1 


1 1 

1 1 

1 1 

1 1 

+ -3 + 


1 1 

1 1 

1 1 

1 1 

1 1 

+ + 






1 1 

1 1 

1 1 

1 1 

+ (0) + 


| Measr | * = 3 


| -Instrument | -INTASC 






1 S.l | 



Figure 4 . FACETS Analysis of Ten INTASC Principles and Three 
Instrument Types 
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Least Squares Means 




LEVEL$ 



Figure 5 . Analysis of Disposition Scores Across Placement 



This figure is a subsample of the pilot where class standing 
was available. During their teacher preparation programs, students 
appear to be acquiring increasing commitment to the skills of 
teaching, as operatinally defined in the INTASC Principles. This 
graph is also confirmed by an ANOVA and the points plotted are the 
least-squared means from that analysis. Also, the variability of 
teacher candidates as final interns is the smallest. The 
variablility of the nontraditional (alternative certification) 
candidates is the greatest. As students progress to final 
internship, they become more consistent and more homogeneous in 
their consistency with INTASC principles. This is evident in the 
standard errors indicated in the points plotted above by the 
brackets above and below each plotted point . 
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Table 5. Results of the FACETS Analysis of BATS, ETQ, and SRA 

Disposition Pilot FACETS Analysis 03-20-2008 12:34:13 
Table 7.2.2 Instrument Measurement Report (arranged by fN) . 



I Obsvd 


Obsvd 


Obsvd 


Fair-M 




Model 


| Infit 




Outfit | 


Estim. | 








I Score 


Count 


Average 


Avrage 


Measure 


S.E. 


| MnSq 


ZStd 


MnSq ZStd | 


Discrm | 


N 


Instrument 




| 2492 


1276 


2.0 


1 . 93 


.71 


.03 


1 1.02 


.4 


1.05 1.3| 


. 98 


2 


ETQ 




10749 


12916 


.8 


.85 


-1 . 74 


.02 


I 1.01 


.3 


1.04 2.1| 


.99 


1 


BATS 




| 2711 


1749 


1.6 


1 . 62 


1 . 04 


.03 


1 . 90 


-3.5 


.93 -2.3| 


1.12 | 


3 


SRA 




I 5317.3 


5313 


7 1.4 


1 . 47 


.00 


.03 


1 . 97 


-.9 


1.01 .4| 




Mean (Count: 3) 




1 3841.8 


5379 


1 .5 


.46 


1.24 


.00 


I .05 


1 . 9 


.06 2.0| 


i 


S 


D. 





RMSE (Model) .03 Adj S.D. 1.24 Separation 46.71 Separation Reliability 1.00 
Fixed (all same) chi-square: 7359.3 d.f.: 2 significance (probability): .00 

Random (normal) chi-square: 2.0 d.f.: 1 significance (probability): .16 



Disposition Pilot FACETS Analysis 03-20-2008 12:34:13 
Table 7.3.1 INTASC Measurement Report (arranged by mN) . 



i 


Obsvd 


Obsvd 


Dbsvd 


Fair-M | 




Model 


i 


Infit 


Outfit | 


Estim. | 






i 


i 


Score 


Count Average 


Avrage | Measure 


S.E. 


i 


MnSq 


ZStd 


MnSq 


ZStd | 


Discrm| 


Nu 


INTASC 


i 


i 


1352 


1552 


. 9 


.69| 


.40 


.05 


i 


1.07 


1.4 


1.17 


4 . 1 | 


.85 | 


9 


Reflection 


i 


i 


1504 


1605 


. 9 


.80 | 


.23 


.05 


i 


1.06 


1.3 


1.26 


5.7 | 


.78 | 


4 


Critical Thinking 


i 


i 


1525 


1606 


. 9 


.82 | 


.19 


.05 


i 


.83 


-3.9 


. 87 


-3.0 | 


1.14 | 


1 


Subject Matter 


i 


i 


1527 


1601 


1.0 


.82 | 


.17 


.05 


i 


1.02 


.3 


1.01 


.2 | 


1.00 | 


10 


Professionalism 


i 


i 


1572 


1603 


1.0 


.86| 


.08 


.05 


i 


. 92 


-1.6 


1.03 


. 5 | 


1.03 | 


8 


Assessment 


i 


i 


1669 


1604 


1.0 


. 94 | 


-.14 


.05 


i 


1.00 


.0 


1.08 


1.5 | 


.97 | 


3 


Diversity 


i 


i 


1680 


1605 


1.0 


. 95 | 


-.16 


.05 


i 


. 97 


-.5 


1.12 


2 . 1 1 


.95 | 


5 


Environment 


i 


i 


1641 


1556 


1.1 


.96| 


-.17 


.05 


i 


1.17 


3.1 


1.21 


3 . 4 | 


.86 | 


7 


Planning 


i 


i 


1697 


1602 


1.1 


. 96 | 


-.21 


.05 


i 


. 86 


-2.8 


.79 


-4.1 | 


1.18 | 


6 


Communications 


i 


i 


1785 


1607 


1.1 


1.04| 


-.40 


.05 


i 


. 81 


-3.7 


.77 


-3 . 9 | 


1.18 | 


2 


Learning 


i 


i 


1595.2 


1594 . 1 


1.0 


.88 | 


.00 


.05 


i 


. 97 


-.6 


1.03 


• 7 | 


i 


Mean (Count: 10) 


i 


i 


117.8 


20.1 


. 1 


,10| 


.24 


.00 


i 


. 11 


2.3 


. 16 


3 . 3 | 


i 


S.D. 


i 


RMSE (Model) .05 


Adj S 


D. .23 


Separation 




4 . 88 


Separation 


Reliability 


. 96 





Fixed (all same) chi-square: 243.4 d.f.: 9 significance (probability): .00 

Random (normal) chi-square: 9.0 d.f.: 8 significance (probability): .34 



Disposition Pilot FACETS Analysis 03-20-2008 12:34:13 
Table 8 . 1 Category Statistics . 

Model = ?,?,?, R 



| DATA | QUALITY CONTROL | STEP | EXPECTATION | MOST | . 5 Cumul . | Cat | 

| Category Counts Cum. | Avge Exp. OUTFIT | CALIBRATIONS | Measure at | PROBABLE | Probabil . | PEAK | 



| Score 


Used 


% 


% i 


Meas 


Meas 


MnSq | 


Measure 


S . E . | Category 


-0.5 | 


from | 


at 


| Prob 


1 o 


426 


14% 


14% | 


-1.40 


-1.48 


1.1 




i < 


-3.34) 


1 


low | 


low 


1 100% 


1 1 


859 


28% 


42% | 


-1.24 


-1.17 


.9 


-2.02 


.06| 


-1.80 


-2.62| 


-2.02 | 


-2.33 


| 40% 


1 2 


1011 


33% 


76% | 


-.93 


-.92 


1.2 


-1.21 


. 04 | 


-.68 


-1.22| 


-1.21 | 


-1.20 


| 40% 


1 3 


603 


20% 


96% | 


-.64 


-.66 


1.0 


-.28 


,05| 


.47 


-,13| 


-.28 | 


-.18 


| 46% 


1 4 


117 


4% 


100% | 


-.13 


-.36 


.8 


1.12 


.09| 


1.87 


1.13| 


1.12 | 


1.11 


| 47% 


1 5 


9 


0% 


100% | 


-.05 


.00 


1.0 


2.38 


,34| ( 


3.63) 


2.84| 


2.38 | 


2.59 


1 100% 



(Mean) (Modal) — (Median) 




Category Information 
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Model = ?,?,?,R (Rating or Partial Credit Scale) 




-5 - 4.5 -4 - 3.5 -3 - 2.5 -2 - 1.5 -1 - 0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 

Measure relative to item difficulty 

0 1 2 3 4 5 

0=Unaware 

l=Receiving 

2=Responding 

3=Valuing 

4=Organizing 

5=Characterizing 



Figure 6. Threshold Analysis of Disposition Scores in FACETS 
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Conclusions and Discussion 

The INTASC dispositions are a complex construct best 
measured using different item structures that presuppose a 
dimensional difficulty for Rasch models. Accreditation 
assessment often presupposes the aggregation of results by 
multiple overlapping frameworks. The importance of this work 
is in its potential to provide a new application for Rasch 
measurement. In this case, the initial study seeks to examine 
the results as one scale that is also informative as mapped 
originally on multidimensional framework and reportable along 
multiple informative scales . 

As suggested by Wight & Stone (1999), the focused scale 
can still be deconstructed and provide useful information if 
the unifying elements were originally applied as a single 
unifying dimension. Even though there appears to be an 
interest in the mathematics of complex Rasch models as 
expressed in Davier & Carstensen' s (2007) Multivariate and 
Mixture Distribution Rasch Models: Extensions and 
Applications , the original mapping of the construct with 
forethought along multiple dimensions appears to allow an 
acceptable Rasch model that also provides reasonable results 
while using traditional calibration methods. 

In this case, the most useful representation comes from 
Figure 4 . The BATS Thurstone scale is a screening instrument 
that is subject to faking and generally utilized with 
beginning education majors who are pre-internship. It appears 
to be the easiest on the FACETS analysis. ETQ is a reflective 
questionnaire that is aimed at teacher candidates near their 
final internship as they have their first experiences as 
teachers. It appears to be the next most difficult. Finally, 
the SRA is a projective instrument (apperception test) that 
would range in scores into the responses expected by 
experienced inservice teachers. The placement of the SRA 
meets that expectation. 

The sample are mostly students at the end of their 
preservice experience. The authors had suggested that a 
typical student would be at the "valuing" level about the time 
of graduation and certification. The mid-range of our sample 
is in the valuing interval, which is expected. The sample 
contained a few beginning graduate students who scored into 
the "organizing" interval, and some lower division students 
were in the "receiving" and "responding" intervals. 

In conclusion, there appears to be ample evidence that 
the original mapped construct is useful for instrument 
development using the Rasch model, even though the overfit and 
underfit with different item types is confounding. The 
category order is consistent empirically with this sample size 
and expected ability. The ten INTASC Principles also 
calibrate in an expected range and order. 

The primary limitation here was the range restriction due 
to a lack of inservice and graduate students, rubric revision 
that was necessary during the pilot due to rater error, and a 
minimal but adequate sample size for pilot work. 
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